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METHOD AND SYSTEM FOR IMAGE PROCESSING FOR PROFILING WITH 
UNCODED STRUCTURED LIGHT 

Field of the Invention 

The present invention relates to apparatus and methods for three 
5 dimensional scanning and particularly, but not exclusively, the invention relates to 
apparatus and methods for determining the order of a sequence of stripes 
captured in an uncoded structured light scanning system. 

Background to the Invention 

10 Scanners, that is devices that can capture an image and convert it into a 

unique set of electrical signals are well known. Similarly 3-dimensionaI (3D) 
scanning systems are known that can image an object to obtain 3D surface 
representations of the object. Fields of application of 3D scanning systems are 
varied and, for example, include medical imaging, reverse engineering, computer 

15 vision, and video and film production. By object it is meant herein any physical 
entity such as a real object, a hidden object located below a surface, but viewable 
using certain kinds of radiation or a physical system such as a fluid flow that is 
not at first sight within the classical meaning of the term object. 

20 The goal of structured light techniques is to measure the shape of three 

dimensional objects using automatic non-contact techniques. Early systems used 
a single stripe or spot of laser light to measure a small part of the object in each 
scan. Now the availability of controlled light output from LCD projectors allows the 
projection of a more complex pattern of light to increase the area measured in a 

2 5 single instantaneous scan. 

(1 ) Single Stripe Systems 

The classic single stripe scanning system is described in the following 

3 0 paper: J .-A. Beraldin, M. Rioux, F. Blais, G. Godin, R. Baribeau, (1992) Model- 

based calibration of a range camera, proceedings of the 11th International 
Conference on Pattern Recognition: 163-167. The Hague, The Netherlands. 
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August 30-September 3, 1992, [Ber92a]. This system provides the profile of one 
"slice" through the target object. In order to build a model of the complete surface 
a number of spatially related profiles must be scanned. To achieve this a 
sequence of scans is captured. For each scan, the target object is moved in 
5 relation to the scanner, or the projected stripe moves in relation to the object, the 
movement being controlled to the same resolution as required by the scanning 
system. As will be understood by those skilled in the art such a single stripe 
system may require an accuracy of 1:20000. 

10 (2) Multi-Stripe Systems 

To avoid the need for accurate mechanisms and in order to speed up the 
acquisition process, a number of stripes can be projected at or substantially at 
the same time and captured as a sequence of stripes in a single frame. Unlike 
with the single stripe systems an accurate rotation estimate is not required and 

is such systems offer free-form imaging by which it is meant imaging without the 
restriction of using a mechanical rotation device or a mechanical rotation indexing 
system. 

3D multi-stripe scanning apparatus and methods are known for 

2 o determining the order of a sequence of stripes captured in an uncoded structured 

light scanning system, i.e. where all the stripes are projected with uniform colour, 
width and spacing. A single bitmap image shows a pattern of vertical stripes from 
a projected source. If the target object is a planar surface then these stripes are 
uniformly spread over the surface in an intuitive manner. However these vertical 
25 stripes are deformed by the surfaces of more complex target objects, such as, for 
example, a target object comprising a physical model of a human head where in 
particular heavier distortion occurs around the nasal area. If a correspondence 
can be determined between the projected stripes and those captured in the 
bitmap, a spatial measurement of the surface can be derived using standard 

3 o range-finding methods. 
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However, it is frequently difficult to determine which captured stripe 
corresponds to which projected stripe, when we attempt to index the captured 
sequence in the same order as the projected sequence. We call this the stripe 
indexing problem, For this reason methods have been devised to uniquely mark 
5 each stripe: 

(1) by colour as described in the paper: C. Rocchini, P. Cignoni, C. 
Montani, P. Pingi and R. Scopigno, (2001) A low cost 3D scanner based on 
structured light, Computer Graphics Forum (Eurographics 2001 Conference 

10 Proa), vol. 20 (3), 2001, pp. 299-308, Manchester, 4-7 September 2001, 
[Roc01a]; 

(2) by stripe width as described in the paper: Raymond C. Daley and 
Laurence G. Hassebrook, (1998) Channel capacity model of binary encoded 

is structured light-stripe illumination, in Applied Optics, Vol.37, No 17, 10 June 
1998, [Dal98a] ; and 

(3) by a combination of both as described in the paper: Li Zhang, Brian 
Curless and Stephen M. Seitz, (2002) Rapid Shape Acquisition Using Color 

20 Structured Light and Multi-pass Dynamic Programming, 1 st International 
Symposium on 3D data processing, visualization and transmission, Padova, 
Italy, June 19-22, 2002, [Zha02a]. 

These and other works highlight the disadvantages of coded structured 
25 light: with colour indexing there may be weak or ambiguous reflections from 
surfaces of a particular colour, and with stripe width variations the resolution is 
less than for a uniform narrow stripe. This last problem can be addressed by 
projecting and capturing a succession of overlapping patterns of differing width 
as disclosed in the paper: Olaf Hall-Holt and Szymon Rusinkiewicz, (2001) Stripe 
3 0 Boundary Codes for Real-Time Structured-Light Range Scanning of Moving 
Objects, proceedings of the Eighth International Conference on Computer Vision 
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(ICCV 2001), July 2001, [HalOla]. However this means that it is not possible to 
measure the surface in a single frame. Single frame or "one-shot" capture is 
desirable because it speeds up the acquisition process, and leads to the 
possibility of capturing moving surfaces. 

5 

Moreover, because of the limits of any colour coding scheme, ambiguities 
will still exist; and stripe width coding is likely to increase the difficulty of 
interpreting shape correctly, such as when occlusions occur. 

10 In summary, existing prior art methods are thus known which uniquely 

encode each stripe, such as by stripe colour or by stripe width, in order to avoid 
ambiguous stripe identification. However, colour coding suffers due to uneven 
colour reflection which is caused by the colour of the physical entity being imaged 
interfering with the colour coded stripes. A major problem with a variable width 

15 coded approach is that it reduces the measured resolution of the physical entity 
being imaged. In view of the problems with the prior art methods there is 
therefore a need to improve structured light scanning systems. Furthermore 
many other imaging methodologies using radiations other than structured light 
experience similar or related problems. Thus there is also a need to overcome 

20 such problems in various imaging methodologies that use radiations other than 
visible light 

Summary of the Invention 

An object of the present invention is to provide a 3D multi-stripe scanning 
25 apparatus and method that enables the stripes obtained from imaging a complex 
object to be accurately indexed (that is correctly identified) without the need for 
using stripes of different widths. 

Another object of the present invention is to provide a 3D multi-stripe 
3 o scanning apparatus and method that enables the stripes obtained from imaging a 
complex object to be accurately indexed without the need for using stripes that 
are colour coded. 
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A further object of the present invention is, in respect of a multi-stripe 
scanning system, to geometrically constrain the relative positions of a physical 
entity being imaged, a radiation projector and a radiation detector so that stripe 
5 indexing can be performed using uniform uncoded stripes. 

Yet a further object of the present invention is to provide a scanning 
method and an apparatus embodying the method that detects a plurality of types 
of occlusions which are defined in accordance with the constrained geometry. 

10 

A further object of the present invention is to provide one or more 
processing routines for performing stripe indexing in such a geometrically 
constrained set-up of a multi-stripe scanner. 

15 Yet a further object of the present invention is to provide an improved free- 

form scanning apparatus and method for use in 3D model building. 

Yet a further object of the present invention is to provide improved 
scanning techniques and apparatus that may be used in a wide variety of imaging 
2 o methodologies that either use stripes that are reflected off of a physical entity or 
perhaps slices through an object (a good example of the latter being CT X-ray 
scanning as used in medical imaging). 

According to a first aspect of the present invention there is provided a 3D 
25 scanning apparatus configured to image a physical entity, said apparatus 
comprising: 

a radiation projector for projecting a plurality of radiation stripes onto said 
physical entity; 

30 

a detector for detecting said striped radiation received from said physical 
entity; and 
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a data storage device for storing said received radiation as a pixellated 
bitmap image; 

5 said apparatus characterised in that said physical entity, said radiation 

projector and said detector are geometrically arranged to constrain the number of 
times that each said received stripe is permitted to occur in a given pixel row of 
said bitmap, said geometrical constraint determining a stored occlusion 
classification comprising a plurality of types of occlusions and said apparatus 
10 additionally comprising an occlusion type processing means configured to utilise 
said classification for detecting said various types of occlusions in said received 
image data. 

Preferably said radiation projector utilises visible wavelength 
15 electromagnetic radiation and said detector is configured to detect visible light. 

Preferably said geometrical constraint restricts said predetermined number 
of times to once per row. 

2 o Preferably said stripes are uniform uncoded stripes. 

Preferably said physical entity is spatially defined with respect to a 
Cartesian coordinate frame (X 0 , Y 0 , Z 0 ); said projector has origin 0 P in a 
Cartesian coordinate frame (X P , Yp, Z P ); and said detector has origin 0 L in a 
25 Cartesian coordinate frame <X L , Y Lj Z L ), and preferably said geometrical 
constraint comprises alignment of said projector and said detector such that 
coordinate axes Z 0 , Z P , Z L , are substantially parallel and coordinate axes X 0 , X P 
and X L are substantially parallel. 

3 0 Preferably said geometrical constraint comprises configuring said 

coordinate axes Z 0 , Z P> Z L , X 0 , X P and Xl to all lie in substantially the same plane. 
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Preferably said geometrical constraint additionally comprises positioning 
said projector so that its origin 0 P iies on said Xl axis of said detector. 

Preferably said geometrical constraint comprises positioning said projector 
5 so that its origin 0 P lies on said X L axis of said detector such that said coordinate 
frame origins 0 P and 0 L lie on the same plane. 

Preferably said same plane is inclined at an angle a to axis Z L . 

10 Preferably said projector projects visible light and said detector comprises 

a video camera having focal length F, said same plane being inclined at an angle a 
to axis Z L and said plane being defined in accordance with the equation: y B = -F 
tana such that a beam projected in said plane is reflected back in the same plane 
and onto the image plane at row -Ftan<x 

15 

Preferably said occlusion type processing means is configured to process 
said obtained image data into an array of peaks of peak pixel data, each said 
peak substantially representing the centre of a said stripe received by said 
detector. 

20 

Preferably said occlusion type processing means is configured to search 
said peaks in said array for discontinuities, said processing means being further 
configured to: 

25 create an occlusion map through identifying and labelling said 

discontinuities as at least being of a first type or of a second type of discontinuity. 

Preferably said created occlusion map is used to define boundaries for 
use by an indexing processing means, said indexing means being configured to 
3 0 index said stripes by using said boundary information and said stored 
classification. 
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Preferably said indexing processing means is configured to process said 
peak pixel data in accordance with steps comprising: 



(a) selecting a start position in said peak pixel data and initialising a stripe 
5 index count for said selected stripe; 

(b) tracing the current stripe in a first direction along said stripe until a said 
first boundary condition is met; 

(c) returning to said start position and tracing said current stripe in the 
opposite direction to said first direction until a said boundary condition is met; 

io (d) increasing the stripe index and moving to the next stripe in a third 

direction, said third direction substantially perpendicular to said first and second 
directions, and repeating steps (b) and (c); 

(e) repeating step (d) until a second boundan/ condition is met; and 

(f) returning to said start position and repeating steps (d) and (e), but 
15 moving in a fourth direction, said fourth direction being substantially 180° from 

said third direction, and decreasing said stripe index each time. 



Preferably said indexing processing means is configured to process said 
peak pixel data in accordance with a flood fill recursive processing routine. 

20 

Preferably said indexed stripes are used to reconstruct a 3D surface of a 
scanned physical entity. 



According to a second aspect of the present invention there is provided, in 
2 5 a 3D scanning apparatus configured to image a physical entity, said apparatus 
comprising a radiation projector for projecting a plurality of radiation stripes onto 
said physical entity, a detector for detecting said striped radiation received from 
said physical entity and a data storage device for storing said received radiation 
as a pixellated bitmap image, a method of imaging characterised in that: 

30 

said physical entity, said radiation projector and said detector are 
geometrically arranged to constrain the number of times that each said received 
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stripe is permitted to occur in a given pixel row of said bitmap, said geometrical 
constraint determining a stored occlusion classification comprising a plurality of 
types of occlusions and said apparatus additionally comprising an occlusion type 
processing means configured to utilise said classification for detecting said 
5 various types of occlusions in said received image data. 

Preferably said method utilises visible wavelength electromagnetic 
radiation. 

10 Preferably said geometrical constraint restricts said predetermined number 

of times to once per row. 

Preferably said stripes are uniform uncoded stripes. 

15 Preferably said physical entity is spatially defined with respect to a 

Cartesian coordinate frame (X 0 , Yo, Z 0 ); said projector has origin 0 P in a 
Cartesian coordinate frame (X P , Y P) Z P ); and said detector has origin 0 L in a 
Cartesian coordinate frame (X Lj Y L , Z L ), and preferably said geometrical 
constraint comprises alignment of said projector and said detector such that 

20 coordinate axes Z 0 , Z P , Z Ll are substantially parallel and coordinate axes X 0 , X P 
and Xl are substantially parallel. 

Preferably said geometrical constraint comprises configuring said 
coordinate axes Z 0 , Z P , Z L , X 0 , X P and X L to all lie in substantially the same plane. 

25 

Preferably said geometrical constraint additionally comprises positioning 
said projector so that its origin 0 P lies on said Xl axis of said detector. 

Preferably said geometrical constraint comprises positioning said projector 
3 0 so that its origin 0 P lies on said X L axis of said detector such that said coordinate 
frame origins 0 P and 0 L lie on the same plane. 
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Preferably said same plane is inclined at an angle a to axis Z L . 

Preferably said projector projects visible light and said detector comprises 
a video camera having focal length F, said same plane being inclined at an angle 
5 a to axis Z L and said plane being defined in accordance with the equation: y, B = -F 
tana such that a beam projected in said plane is reflected back in the same plane 
and onto the image plane at row -F tana ^ 

Preferably said occlusion type processing step processes said obtained 
10 image data into an array of peaks of peak pixel data, each said peak substantially 
representing the centre of a said stripe received by said detector. 

Preferably said occlusion type processing step comprises the steps of: 

15 searching said peaks in said array for discontinuities; and 

creating an occlusion map through identifying and labelling said 
discontinuities as at least being of a first type or of a second type of discontinuity. 

20 Preferably said occlusion map is used to define boundaries for use in 

indexing said stripes, said indexing using said boundary information and said 
stored classification. 

Preferably said indexing step is configured to process said peak pixel data 
25 in accordance with steps comprising: 

(a) selecting a start position in said peak pixel data and initialising a stripe 
index count for said selected stripe ; 

(b) tracing the current stripe in a first direction along said stripe until a said 
3 o first boundary condition is met; 

(c) returning to said start position and tracing said current stripe in the 
opposite direction to said first direction until a said boundary condition is met; 
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(d) increasing the stripe index and moving to the next stripe in a third 
direction, said third direction substantially perpendicular to said first and second 
directions, and repeating steps (b) and (c); 

(e) repeating step (d) until a second boundary condition is met; and 

5 (f) returning to said start position and repeating steps (d) and (e), but 

moving in a fourth direction, said fourth direction being substantially 180° from 
said third direction, and decreasing said stripe index each time. 

Preferably said indexing step comprises processing said peak pixel data 
10 in accordance with a flood fill recursive processing routine. 

Preferably said indexed stripes are used to reconstruct a 3D surface of a 
scanned physical entity. 

15 According to a third aspect of the present invention there is provided a 3D 

scanning apparatus configured to image a physical entity, said apparatus 
comprising: 

a radiation projector for projecting a plurality of radiation stripes onto said 

2 o physical entity; 

a detector for detecting said striped radiation received from said physical 
entity; and 

25 a data storage device for storing said received radiation as a pixellated 

bitmap image; 

said apparatus characterised in that said physical entity, said radiation 
projector and said detector are geometrically arranged to constrain the number of 

3 0 times that each said received stripe is permitted to occur in a given pixel row of 

said bitmap. 
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According to a fourth aspect of the present invention, there is provided in a 
3D scanning apparatus configured to image a physical entity, said apparatus 
comprising a radiation projector for projecting a plurality of radiation stripes onto 
said physical entity, a detector for detecting said striped radiation received from 
5 said physical entity and a data storage device for storing said received radiation 
as a pixellated bitmap image, a method of imaging characterised in that: 

said physical entity, said radiation projector and said detector are 
geometrically arranged to constrain the number of times that each said received 
o stripe is permitted to occur in a given pixel row of said bitmap. 

According to a fifth aspect of the present invention there is provided a 3D 
scanning apparatus configured to image a physical entity, said apparatus 
comprising: 

5 

a radiation projector for projecting a plurality of radiation stripes onto said 
physical entity; 

a detector for detecting said striped radiation received from said physical 
o entity; and 

a data storage device for storing said received radiation as a pixellated 
bitmap image; 

5 said apparatus characterised in that said physical entity, said radiation 

projector and said detector are geometrically arranged so that each said received 
stripe is permitted to occur only once in a given pixel row of said bitmap. 

According to a sixth aspect of the present invention there is provided a 3D 
o scanning apparatus configured to image a physical entity using a plurality of 
stripes, said apparatus comprising: 
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a radiation projector for projecting radiation onto said physical entity; 

a detector for detecting said striped radiation received from said physical 
entity; and 

5 

a data storage device for storing said received radiation as a pixeilated 
bitmap image, 

said apparatus configured in accordance with a predefined geometrical 
10 arrangement, wherein: 

said predefined geometrical arrangement comprises a constraint such that 
each said received stripe can only occur a predetermined number of times in a 
pixel row of said bitmap. 

15 

Brief Description of the Drawings 

For a better understanding of the invention and to show how the same 
may be carried into effect, there will now be described by way of example only, 
20 specific embodiments, methods and processes according to the present 
invention with reference to the accompanying drawings in which: 

Figs. 1 and 2 schematically illustrate a scanning apparatus comprising a 
projector and a camera as set up and constrained in accordance with the present 
25 invention: Fig, 1 is a view along the Y axis and Fig. 2 is a view along the X L axis, 
Fig. 2 illustrating the so called "Common Inclination Constraint" of the present 
invention; 

Fig. 3 schematically illustrates another view or representation of the 
3 0 Common Inclination Constraint illustrated in Fig. 2; 
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Fig. 4 schematically illustrates, in accordance with the present invention, 
steps involved in processing the pixellated bitmap array obtained using a 
scanner as configured in accordance with the present invention and, in 
particular, includes the step of creating a peak data array; 

5 

Fig. 5 schematically illustrates, in accordance with the present invention, 
examples of moving around the peaks data of a peak data array of the type 
illustrated in Fig. 4; 

io Fig. 6 schematically illustrates, in accordance with the present invention, 

the effect of a preferred embodiment of an indexing processing routine (the 
stripe tracer algorithm) to index the stripe peaks; 

Fig. 7 schematically illustrates, in accordance with the present invention, 
is the effect of a preferred embodiment of an indexing processing routine (the 
flood filler algorithm) to index the stripe peaks; 

Fig. 8 schematically illustrates known Projector and Camera (detector) 
type Occlusions required for understanding the present invention in relation to 
2 o classifying occlusion types; 

Fig. 9 schematically illustrates types of occluded areas with respect to a 
projector and a camera required for purposes of understanding the present 
invention, the figure showing that the viewpoint for a camera occlusion is the 
2 5 camera origin and the viewpoint for the projector occlusion is the projector origin; 

Fig. 10 schematically illustrates, in accordance with the present invention, 
a classification for four classes of typical occlusions in relation to the associated 
stripe disturbances thereby caused; 

30 
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Fig. 1 1 schematically illustrates, in accordance with the methods of the 
present invention, an example of connecting occlusion points so as to enable 
correct stripe indexing; 

5 Fig. 12 illustrates patches of a test surface obtained using the indexing 

processing routines as implemented in accordance with the present invention. 
Patches of a template and four tests are shown, tests 1 and 2 being of the stripe 
tracer routine and tests 3 and 4 being of the flood filler routine (see table in Fig. 
14); 

10 

Fig. 13 (a), the figure on the left, schematically illustrates typical 
connectivity errors at points A, B and C; 

Fig. 13(b), the figure on the right, schematically illustrates, in accordance 
is with the occlusion detection methods and the apparatus as configured in 
accordance with the present invention, correction of the connectivity errors 
illustrated in Fig. 13 (a); and 

Fig. 14: schematically illustrates a table detailing numerical analysis of 

2 o correspondence between template and automatic models, with and without the 

occlusion boundaries. The starting point is at (0,0). 

Detailed Description 

There will now be described by way of example a specific mode 
25 contemplated by the inventors. In the following description numerous specific 
details are set forth in order to provide a thorough understanding. It will be 
apparent however, to one skilled in the art, that the present invention may be 
practiced without limitation to these specific details. In other instances, well 
known methods and structures have not been described in detail so as not to 

3 o unnecessarily obscure the description. 
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To avoid the problems associated with 3D multi-stripe imaging systems 
of the type discussed earlier, in accordance with the present invention, the 
projection of the radiation being used is simplified as a uniform stripe pattern, 
and, as described in detail below, novel methods for correctly indexing the 

5 stripes are developed, including provision of a common inclination constraint for 
constraining the geometry of the scanning apparatus and provision of an 
occlusion classification to be used in indexing the stripes so as to correct 
wrongly connected or unconnected stripe portions. Detailed definitions of 
patches and the continuity of stripes are given and experimental results 

0 concerning the success of the methods employed are provided. Using the 
methods and apparatus as configured in accordance with the present invention 
eliminates the need for coding, and reduces the accuracy required of the 
projected pattern. Furthermore by dealing with stripe continuity and occlusions 
in accordance with the methods of the present invention, general methods are 

5 provided which have relevance to many structured light problems and various 
kinds of other imaging methodologies. 

In accordance with the following disclosure an examination is made as to 
how far the correspondence problem can be solved with uncoded stripes, where 
o the correct order of stripes in the captured image is determined by various 
preferred processing routines that have been configured in accordance with the 
present invention. Each captured frame will therefore provide a complete data 
model for a patch on the surface of the target object 

5 In part 1 the system is described, showing the dependence between the 

stripe index and the measurement of a surface point, and a preferred "common 
inclination constraint" geometry is defined. In part 2 the continuity of stripes and 
the boundaries between continuous patches are defined, and two algorithms for 
implementation as processing routines are designed each of which indexes a 

o sequence of corresponding stripes. In part 3 an occlusion classification is given 
to improve the validity of the boundaries, and a further connectivity algorithm is 
described. In part 4, the index is compared to a template sequence, created by 
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hand using prior knowledge to ensure an exact correspondence with the 
projected sequence. The results of implementing both types of processing 
routine, with and without dealing with occlusions, are presented. 



5 1 . DESCRIPTION OF A PREFERRED EMBODIMENT OF THE SYSTEM 

The scanning apparatus is schematically illustrated in plan view in Fig. 1 . 
Evenly spaced, vertical (i.e. parallel to the Y axes) stripes are projected which 
intersect the Xq axis, and the distance between intersections is W. A point s = 
(x,y,z) on the surface of the target object reflects a beam (shown dotted) in stripe 

io n through the camera lens at Oi and onto the image plane. This light is sensed at 
(/?,v) in the camera CCD array, measured to an accuracy better than one pixel by 
the subpixel estimator defined later in this section. The relation between the CCD 
pixel array and the spatial image plane is given by x B = hCF, y B = vCF, where F is 
the focal length of the camera and CF is the spatial size of one square pixel. P 

15 and D are the distances from the origin X 0 Y 0 Zo to the centres of the projector and 
camera lens respectively. 9 is the angle between the Z axes of the camera and 
projector. 

The parameters of s are given by the scanning function scan {h,v,n) = (x,y,z) 



25 



Wn (2.1) 



x = Wn~z — 
P 

2 0 y = vC(zw&$-D) (2-2) 

DhC + Wnjcos 6 + hCsin 0) (2.3) 
hC cos 6 - sin 9 + ~ (cos Q + hC sin 0) 

where h and v give the position of the sensing pixel and n is the index of the 
stripe containing the sensed beam. The constants W, P, D, C and 9 are found by 
calibration. Note that, by construction, P is never equal to zero. 

Hence it can be seen that successful measuring of the target surface is 
dependent upon correctly determining the index n of each stripe appearing in the 
image. 
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Fig- 2 schematically illustrates the apparatus of the system of Fig. 1 as 
viewed "sideways", i.e. down the Xl axis. In order that parametric equations 2.1, 
2.2 and 2.3 are correctly formulated, the projector and camera are aligned 
"horizontally", so that axes Z 0 , Z P , Z L X 0 , X P and X L are all in the same plane. 

5 

In the preferred embodiment these axes are axes of Cartesian co-ordinate 
frames, but as will be understood by those skilled in the art, other forms of co- 
ordinate frames could be used. In the preferred set of co-ordinate frames, the 
co-ordinate axes Z 0 , Z P , Z L are substantially parallel and co-ordinate axes X 0 , X P 

1 o and Xl are substantially parallel. 

In the best mode contemplated, the geometrical constraint comprises 
configuring the co-ordinate axes Z 0 , Z P , Z L , X 0 , X P and Z L to all lie in substantially 
the same plane. 

15 

THE COMMON INCLINATION CONSTRAINT 

The system is further constrained by positioning the projector so that its 
origin 0 P lies on the X L axis of the camera. Fig. 2 shows that 0 P and 0 L will now 

2 o both lie on a plane y B = -F tana, inclined at angle a to Z L . A beam projected in this 

plane to surface point s will be reflected back in the same plane and onto the 
image plane at "row" -F tana . If it is assumed that any beam can only be 
reflected at one surface point, and that any stripe can only contain one beam at a 
specific inclination, it follows that each stripe can be sensed at only one position 
25 in any "row" in the image plane. This is only true when the system is constrained 
so that the projected and reflected line of a beam always shares a "common 
inclination", hence the name common inclination constraint 

Another view of this constraint is illustrated schematically in Fig. 3. In Fig. 3 

3 0 consider beams from 0 P striking the surface at S? and S2. Lines Opto and 0 P 

to S 2 are projected through the lens at Ol and onto the image plane / as lines 0 P 
to Si' and 0 P to S 2 '. If 0 P follows the arrow to where line Opto 0 L is parallel to 
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image plane /, its "image" Op will move towards infinity, and all lines Opto S n r will 
be parallel. Therefore, if all stripes are considered as a bundle of beams 
emanating from the unique projection point O p , then it follows that every beam 
path will appear to be parallel when viewed from the image plane. 

5 

A 3D scanning apparatus configured in accordance with the present 
invention to image a physic entity using a plurality of stripes may thus comprise a 
radiation projector for projecting radiation onto the physic entity and a detector for 
detecting the stripes radiation received from the physical entity. The detector, 

10 and structured light systems, suitably comprises a video camera and the radiation 
projector suitably comprises a structured light project. The detector itself or a unit 
attached thereto is configured to store the received radiation as a pixellated 
bitmap image and, as discussed above, the apparatus is configured in 
accordance with a pre-defined geometrical arrangement such that the 

15 arrangement puts a constraint on stripes received in that they can only occur a 
pre-determined number of times in a pixel row of the bitmap. In the preferred 
embodiment described above the received stripe may occur only once in a pixel 
row of the bitmap. This enables stripes to be used in scanning that are uniform 
uncoded stripes. 

20 

In the best mode the invention utilizes a CCD sensor. However other 
suitably configured light sensing devices could potentially be used as will be 
understood by those skilled in the art. Whatever type of sensing device is used 
the radiation projector, the physical entity being imaged and the radiation detector 

25 are geometrically arranged such that the arrangement of these devices 
constrains the number of times that each received radiation stripe is permitted to 
occur in a given row of the bitmap of the received radiation. The normal 
subdivision of pixels into rows is a convenient subdivision of pixels in the sensor 
to utilise. However other predefined sets of pixels could be used. Thus in this 

3 o sense the meaning of the word "row" is not to be considered as limited to either a 
horizontal array of pixels or to a consecutive sequence of pixels. Depending on 
the geometrical arrangement adopted a vertical array (a column), for example, 
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could be used. As will be understood from Figs. 1 to 3 and the associated 
description thereof the term "row" is to be interpreted as including any set of 
pixels in the sensor device such that each pixel in the set is coincident with the 
plane which is normal to the plane of the central radiated stripe. In the best mode 
contemplated the pixels in a given such set comprise a consecutive sequence of 
pixels in the CCD sensor and this is conveniently provided by a conventional row 
of pixels (or a column if the apparatus is configured to work such that a vertical 
sequence of pixels constitutes the required arrangement of a set of pixels). 

PROCESSING THE IMAGE DATA 

A suitably configured scanning system as configured in accordance with a 
preferred embodiment of the present invention uses a standard monochrome 
video camera synchronised to the PAL standard, at a bandwidth of 5.5 MHz. The 
data is presented to the system processors as a C x R array A B of discrete 
brightness levels where a B e[0, 255], and the array is situated in the sensing 
plane of the camera. 

Fig. 4 schematically illustrates, in accordance with the present invention, 
steps involved in processing the pixellated bitmap array obtained using a scanner 
as configured in accordance with the present invention and, in particular, includes 
steps for creating a peak data array. Referring to Fig. 4, from the bitmap array A Q 
where f{c,r) = a e , a peak array A P is created of local horizontal maxima, i.e. peaks 
at the centre of each stripe, where: 




TRUE if f(c-l,r)<f(c,r)<f(c + l,r) 
FALSE otherwise 



(2.4) 
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The indexing algorithms then use the peaks array Ap to label each peak 
with a stripe index array >A/ = CxRxN where n is either a stripe index or "NULL" 
(shown blank). To provide h, v and n for the scanQ function, a further data 
type / = H x V x N is created using a subpixel estimator spe(c,r) = (fyvj with a 
5 design similar to those used by 

R. B. Fisher and D. K. Naidu (1996) A Comparison of Algorithms for 
Subpixel Peak Detection, in Sanz (ed.) Advances in Image Processing, 
Multimedia and Machine Vision, Springer-Verlag, Heidelberg (fis 96a). 

10 

2. STRIPE INDEXING 

The discrete bitmap image on the sensing plane is a square tessellation of 
pixels, each pixel representing a square with four corners and four sides. 

15 Definition 1 Pixels P and Q are called direct neighbours (d-neighbours) if 

they share a side, and indirect neighbours (hneighbours) if they share one and 
only one comer. 

As all pixels in the captured image are situated in a two dimensional 
20 Euclidean plane, then each pixel may have maximally eight neighbours (four 
direct and four indirect) as seen in Fig. 5, at position 4. 

We now refer to the peaks array A P in which pixels are marked as TRUE if 
they are peaks (shown grey in Fig. 5). 

25 

Definition 2 A northern neighbour (n~neighbour) of a current peak is 
defined as a d- or i- neighbour in the row incremented by one. A southern 
neighbour (s-neighbour) of a current peak as a of- or /- neighbour in the row 
decremented by one. 

30 
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Definition 3 A western neighbour (w-neighbour) is defined as the nearest 
peak in the same row which has a lesser column index. An eastern neighbour fe- 
neighbour) is defined as the nearest peak in the same row which has a greater 
column index. 

5 

The maximum distance to the "nearest" peak will be defined by an 
estimation of the greatest likely separation of two consecutive stripes. The 
definition of i-neighbours assumes that the stripe cannot move laterally by more 
than one pixel as it moves up or down, i.e. that the "slope" of the stripe is never 
10 less than 45°. This will not be true in practice but, in common with our general 
approach it will create extra boundaries rather than wrongly connected stripes. 

Fig. 5 schematically illustrates, in accordance with the present invention, 
examples of moving around peak data of a peak data array of the type illustrated 
15 schematically in Fig. 4. Fig. 5 shows, relative to peak 1, its n-neighbour A/, its s- 
neighbour S, its w-neighbour l/l/and its e-neighbour E 

Definition 4 A continuous stripe is a sequence of distinct peaks {P 1t P 2 , . - 
. , Pr} such that each peak in the sequence {P 1f P 3 , . . . , Pm) has one and only 
2 o one northern neighbour. 

Consequently sequence {P 2 , P 3 , . . ■ , Pn} has one and only one southern 
neighbour. 

25 Definition 5 A patch is a subset of A P , maximally possible, Le. its peaks 

can be arranged in a maximal number of successive continuous stripes. 

Boundary peaks are created by a set of conditions which are tested at 
each position in the peaks data. The test assumes that a valid position must have 
30 valid adjacent positions. In Fig. 5, starting at position 1, position NE can be found 
by moving north and east, or by moving east and north. Similar tests are 
performed in the south-east, north-west and south-west directions. From position 
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2 moving north and east arrives at a different position from moving east and 
north. Position 2 is therefore marked as a boundary, caused by the disconnected 
peak at position 3. 

5 In order to index the stripe peaks and mark boundaries, two preferred 

embodiments of algorithms for implementation in indexing processing routines 
have been designed: the stripe tracer and the flood filler. 

Fig. 6 schematically illustrates, in accordance with the present invention, 
10 the effect of the stripe tracer processing routine used to index the stripe peaks. 
The sequence of operations for the stripe tracer algorithm, shown in Fig. 6 is: 

1 . Find a start position in the peaks data space. Set the stripe index to zero. The 
start point can be found by hand or automatically; 

15 

2. Trace the current stripe northwards until a boundary condition is met; 

3. Return to start and repeat 2 moving southwards; 

4. Increase the stripe index, move to the next peak eastwards and repeat steps 
20 2 and 3; 

5. Repeat step 4 until boundary condition is met; and 

6. Return to start and repeat steps 4 and 5 moving westwards, decreasing the 
2 5 stripe index each time. 

The disadvantage of this process is that once a boundary has been 
reached, the stripe will not be "picked up" again, as can be seen from the lower 
part of the two middle stripes. Therefore some valid stripes in the continuous 
30 surface will be lost. To address this problem a second algorithm for 
implementation in a data processing routine has been devised to perform a more 
thorough search for valid stripes: the flood filler. 
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The flood fill recursive function is a known class of algorithm used in 
graphics software. It has been extensively adapted for use in implementation of 
the present invention as regards passing parameters of stripe, row, heading and 
5 index. In pseudocode it may be expressed as follows: 

flood(stripe, row, heading, index) { 
if(heading==NORTH) goNorth(); 
if(heading==SOUTH) goSouthQ; 

1 o if(heading==EAST) goEast(); index++; 

if(heading==WEST) goWest(); index-; 

if(boundary || alreadylndexed) return; 

15 indices[stripe][row] = index; 

flood(stripe, row+1, NORTH, index); 
flood(stripe, row-1, SOUTH, index); 
flood(stripe+1, row, EAST, index); 

2 0 f!ood(stripe~1 , row, WEST, index); 

Fig. 7 schematically illustrates in accordance with the present invention the 
effect of the second preferred embodiment of an indexing processing routine 
(that of the flood filler detailed above) to index the stripe peaks. 

25 

The flood filler will move north if boundary conditions allow, otherwise east, 
otherwise south, or finally west. If it cannot move in any direction, the current 
floodQ function is taken from the stack revealing the parameters of the previous 
position and this is repeated until a previous position is returned to which has a 

3 0 valid move. In Fig. 7 when the algorithm arrives at 5 it cannot move, and peels 

back until it arrives at previous position 6, whence it can move west to 7. A 
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processing routine using the flood filler approach will index the stripes missed in 
Fig. 6, the stripe tracer. 

3. DEALING WITH OCCLUSIONS 

5 

A scanning apparatus configured in accordance with the geometrical 
constraint defined earlier provides an occlusion classification usable by the 
system. Various types of occlusions may be stored in electronic memory as a 
classification that may be referred to and utilised by an occlusion type processing 
l o means configured to detect the occlusions in the received image data. 

The tests in Section 4 (see later) show that connectivity algorithms 
implemented as processing routines produce some indexing errors, typically by 
connecting peaks which are from different stripes. These errors are often caused 
15 by occlusions. An occlusion is an obstruction, and in a scanning system 
configured in accordance with the present invention there are two types: a 
projector occlusion and a camera occlusion. 

Fig. 8 schematically illustrates known projector and a camera (detector) 
20 type occlusions required for understanding the present invention in relation to 
classifying occlusion types. 

In Fig. 8 (left), four stripes radiate from the projector origin at P, and 
illuminate the surface at positions 1, 2, 3 and 4. In the Figure we draw a dashed 

25 line from P which grazes the surface at point 0 Nl and continues to point 0 Fi 
where the line intersects the surface at a second point. This line we call a 
supporting line, which for a smooth surface will be tangential. The part of the 
surface between On and 0 F is an occluded area, shaded from the projection 
point P. We call 0 N a near occlusion point and 0 F a far occlusion point Note that 

3 0 an actual beam can only strike the surface at or close to one of these two points. 



WO 2005/076214 



PCT/GB2005/000344 



-26- 

Fig. 8 (right) again shows four stripes cast onto a surface, and a 
supporting line from the camera origin C to near occlusion point 0 N and far 
occlusion point Op. Here the occluded area is that part of the surface which is 
hidden from the camera view. Therefore, importantly, the viewpoint for the 
5 camera occlusion is the camera origin, and the viewpoint for the projector 
occlusion is the projector origin. 

Definition 6. A near occlusion point is a point on the surface of the target 
object, such that a straight line from that point to the viewpoint is supporting to the 
o surface at the near occlusion point 

Definition 7. A far occlusion point is a point on the surface of the target 
object, such that a straight line from that point to the viewpoint will support the 
surface at a near occlusion point 

5 

OCCLUSION BOUNDARIES 

Fig. 9 provided for the purposes of understanding of the present invention 
schematically illustrates types of occluded areas with respect to a projector and a 
o camera. Fig. 9 demonstrates that, in accordance with the geometrical 
constraints of the present invention, that the view point of a camera occlusion is 
the camera origin and the view point of the projector occlusion is the projector 
origin. 

5 Fig. 9 (left) shows a target object, a circular button protruding from a fiat 

surface, lit by two light beams in the same stripe plane, where the angle (3 
between the beams is very small. One beam strikes the surface at a near 
occlusion point 0 N , denoted by a ring, the other at a far occlusion point 0 F , 
denoted by a black disc. We see an occluded area shaded grey which is defined 

o by a set of near and far occlusion points (rings and disks) lying on the occlusion 
boundary. This shaded area is the result of a projector occlusion, but there is a 
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second occluded area to the right of the dotted line, caused by a camera 
occlusion, and this hides some of the projected light from the projector view. 

Fig. 9 (right) shows the same target object, viewed in the image plane of the 
5 camera, when the system is set up with the Common Inclination Constraint. We 
recall that with this constraint the beams appear to run in parallel directions, 
which means that Of will be translated laterally from 0 N but will be vertically very 
close (a function of the angle fi). 

10 OCCLUSIONS OF PROJECTED STRIPE PATTERNS 

We can extend these observations to look at stripe patterns on a target 
surface. Fig. 10 schematically illustrates a classification for four classes of typical 
occlusions in relation to the associated stripe disturbances thereby caused, the 

15 classification representing the possible occlusion types that may arise in a 
scanning system as configured in accordance with the present invention. Fig. 10 
shows the target object of Fig. 9, this time lit by a stripe pattern, indexed from 1 to 
7. Position c repeats the situation in Fig. 9, where we can assume that two 
adjacent beams from stripe 2 strike the surface close to an occlusion and are 

2 o translated laterally. Similar situations occur at b, d and e. 



In Fig. 10 (right) parts of the object are magnified to pixel level, and four 
typical situations are classified: 

at (b) a low projector occlusion (l.p.o.); 
25 at (c) a high projector occlusion (h.p.o.); 

at (d) a high camera occlusion (h.c.o.); and 
at (e) a low camera occlusion (Leo.). 



30 
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EXTENDING CONNECTIVITY TO DEAL WITH OCCLUSIONS 

We can now add further rules to our connectivity algorithms, using the 
cases derived from Fig. 10. Firstly, a new data set is created from the peak array 
5 Ap, called the occlusion array, Ao with elements: 



N if -4p(c,r)) , , 

D if ^0(c-l,r-l)v /7(c,r-l)v p(c + l,r~l)) \*~°f 

C otherwise 



Thereby peaks, i.e. TRUE pixels, are labelled as disconnected going up 
10 (U), disconnected going down (D), or connected (C). Non peaks are denoted as 
N. 



Fig, 1 1 (a) shows a practical example of connectivity, using A 0 and the 
cases shown in Fig. 10. From the D peak at position 1 we look for U peak 
15 complying with case b, c, d or e. Note that we can cross fully connected stripes in 
our search. 



In Fig. 11 (b) we have found our U peak at position 2, corresponding to a 
high camera occlusion (case d). The occlusion boundary is drawn as a dotted line 
20 connecting the D peak with the U peak. This occlusion line now crosses a 
seemingly connected stripe, and we therefore know that this is a falsely 
connected stripe, and that there must be a break approximately in the region of 
the occlusion line. We therefore mark U and D peaks at positions 3 and 4. 



25 We can see intuitively that a correct indexing should now give us three 

stripes, where the stripe ending at 1 resumes at 3, and the stripe broken at 4 
resumes at 2. The indexing algorithms should find these connections now that 
the new occlusion boundaries are known, and preliminary results show some 
success. 
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4. EXPERIMENTAL RESULTS 

In this section we present the results of implementation of both algorithms. 
As a target object we use a sculpted head, 200mm. high. We project 200 black 
5 and 200 white stripes into the camera viewing volume, which equates to a 
spacing of 6 pixels between white peaks. The theoretical limit is 2 pixels, i.e. one 
white pixel and one black pixel, although this would result in unresolvable spacing 
at surfaces which are oblique to the camera. The results use a maximal test set 
within this 6 pixel spacing, and future work will reduce the spacing minimum. 

10 

The above procedures will produce patches of surface considered by the 
system to be valid. To measure the success of the two algorithms we compare a 
produced data set with a previously created template. To create the template, a 
copy of the bitmap image is marked by hand so that the system will provide a 
15 data set which is judged to be correct by eye using prior knowledge of the target 
surface. 

Fig. 12 illustrates resultant patches obtained for a test surface. The patches 
were obtained using the aforementioned indexing processing routines as 

2 o implemented in accordance with the present invention. Patches of a template 

and four tests are shown, tests 1 and 2 being of the stripe tracer routine and test 
3 and 4 being of the flood filler routine. 

It can be seen from Fig. 12 that the flood filler (tests 3 and 4) covers a larger 
25 patch than the stripe tracer (tests 1 and 2). This observation is evaluated in the 
table shown in Fig 14. The tests show that errors occur due to miscorresponding 
and noncorresponding peaks. 

We recall from Section 1 the array Ai of index values (either a stripe index or 

3 0 "NULL") for a pixel at position (c,r) in the bitmap image. Function t(c f r) gives the 

index of pixel (c,r) in the template, and a(c,r) the index of the pixel in the same 
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position in the index array produced by the automated algorithms. We then 
produce the following sets: 

♦ C, "corresponding indexed peaks", i.e. the number of elements where the 
5 peaks have the same index. Peak (c,r)e C such that: 

(t(c f r) = a(c,r)) a (a(c,r) * NULL) 

♦ M, "miscorresponding indexed peaks", i.e. the number of elements where the 
peaks have different indices. Peak (c,r)eM such that: 

1 0 (t(c, r) * a{c, r)) a (a(c, r) * NULL) a (f (c, r) * NULL) 

♦ X, "noncorresponding indexed peaks", i.e. the number of elements where a 
false peak is found. Peak (c,r)eX such that: 

(a(c 9 r) * NULL) a (/(c, r) - NULL) 

15 

These comparisons are presented in the table shown in Fig. 14. It can be 
seen that the stripe tracer (test 1) gives a smaller total patch (21347 peaks) but 
with much greater correspondence to the template than for the flood filler (test 3) 
which covers a greater area (45641 peaks) but with many more differences from 
2 o the template (51 88 + 14333 peaks). 

These tests are then repeated (tests 2 and 4) when the occlusion 
boundaries are added to the algorithm conditions. A common connectivity error is 
seen in Fig. 13 (a), where stripes are shown as connected at A, B and C when, 
2 5 with prior knowledge, we know that a boundary exists, caused by an occlusion. 
These connectivity errors at A, B and C are corrected using occlusion detection 
and the results are shown in Fig. 13 (b) with the "correct" indexing at A, B and C. 
The "correctness" is measured in the tests tabulated in the table of Fig. 14. 

30 Figure 14 details, in table form, numerical analysis of correspondence 

between template and the automatic models, with and without the occlusion 
boundaries. The starting point is at (0,0). 
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It can be seen from the table in Fig. 14 that for the stripe tracer the mis- 
and noncorresponding peaks are reduced to zero when occlusion boundaries are 
included. In addition, the occlusion boundaries prevent the more pervasive flood 
5 fill algorithm from finding non-corresponding peaks (14333 peaks reduced to 31). 

5. CONCLUSIONS 

Using the apparatus and methods of the present invention it is clear from 
10 the results given above that bounded patches can be created which accurately 
model the surface of part of a target object, using a uniform stripe scanning 
system. To achieve this we have defined system constraints such as the common 
inclination constraint, to simplify the indexing algorithms. We have then defined 
stripe continuity, patches and boundaries on the target surface. We have shown, 
15 in the tests against a template, that each of these factors contributes to the 
successful scanning of the target object 

An open issue is the complexity of the algorithms. The estimated upper 
bound of the stripe indexer algorithm is 0(n z ) where n is the maximum number of 
20 rows in the bitmap image. This value can be improved. The complexity of the 
flood filler algorithm is under investigation. 

A comparison of the flood filler and stripe tracer algorithms shows that 
greater correspondence can be achieved with an algorithm which has more 
25 constrained boundary tests and therefore creates a smaller patch. Our current 
algorithms follow the principle of "greater constraint and smaller patches". 

The addition of boundaries deduced from the likely position of occlusions 
has been added to the indexing algorithms, which have further increased the 
3 o contribution of correctly corresponding peaks, while reducing the overall patch 
size. These results are important for solving the indexing problem and provide a 
robust and significant contribution to the creation of accurate patches. 
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As regards to the preferred embodiment of the present invention, to 
summarise the following main stages are involved: 

• The apparatus is set up in a specific geometry so that in the video 
image each stripe can appear only once in a particular row. This "rule" 
is then fed into the occlusion classified; 

• Occlusion classification 

Occlusions (such as areas around an imaged human nose (are one of 
the main problems. In accordance with the present invention a unique 
classification is devised which involves the common inclination 
constraint rule. From this four occlusion classes are obtained which can 
be observed in the stripe image; 

« Occlusion Map 

The occlusion classes obtained are then mapped into a new image 
space and the resultant image data is fed into the indexing algorithms; 

• Indexing Algorithms 

The stripe image is then analysed again, and using the fact that the 
occlusions have been marked (for example on the nose) a stripe 
indexing routine is then executed. The stripe indexing routine searches 
the image for connected stripes and presents a map of index stripes 
(see Figs 6 and 7) which can then be used to measure the surface. 

As has been indicated previously, these 4 processes are consequential 
upon each other. 
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It will be understood by those skilled in the art that the methods and 
techniques described herein may find useful application in a wide variety of 
imaging systems. Thus the methods and techniques and apparatus described 
are not to be considered as limited to structured light scanning systems. Thus 
5 scanning systems that use other types of radiation may find useful application of 
the methods and techniques described herein. 



It is also to be understood by those skilled in the art that many of the 
processing functions described in relation to the invention may be implemented in 
1 o software (as computer programs) or as hardware or firmware. 



